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Background: In Korea, clinical performance examination (CPX) has been included in license examination for medical 
doctors since 2009 in order to improve clinical performance of medical students. This study aimed to evaluate the 
contribution of CPX to medical education. 

Methods: Clinical competency in the differential diagnosis of secondary headache was compared between the incoming 
interns in 2009 unexposed to CPX and the incoming interns in 2010 exposed to CPX, using the data of patients who 
visited the emergency department due to headache (181 patients seen by 60 CPX non-exposed interns and 150 patients 
seen by 50 CPX-exposed interns). We obtained the data by reviewing electtonic medical records and nominal lists of 
doctors. Clinical competency was assessed by sensitivity and specificity between the diagnostic impression by interns 
and the final diagnosis. The association between CPX exposure and clinical competency in secondary headache 
diagnosis was evaluated using multiple logistic regression analysis. 

Results: When we assessed clinical competency on the basis of all listed diagnostic impressions, sensitivity and specificity 
were 67.9% and 80.0%, respectively, for headaches seen by CPX-exposed interns, and 51.7%, and 71.7%, respectively, for 
headaches seen by CPX non-exposed interns. Multivariable adjusted logistic regression analysis showed exposure to CPX 
was not associated with increased competency for identifying secondary headache. 

Conclusion: Exposure to CPX as a part of the medical license examination was not effective for the improvement of clinical 
competency of interns in identifying secondary headache. 
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INTRODUCTION 



Headache is one of the most common symptoms with great 
disease burden worldwide and thus, primary care physicians 
frequently encounter patients with headache in daily practice. 1,2) 
Headache develops from various causes, some of which pertain 
to fatal secondary headache. 3 ' Therefore, it is very important for 
physicians to accurately identify secondary headache, 3 5> which 
raises the importance of training medical students to make an 
accurate differential diagnosis of headache. 6 ' 
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In Korea, the clinical performance examination (CPX) 
was included in the license examination for qualified medical 
doctors in 2009 for the first time in order to improve clinical 
competence of medical students. 7 ' Accordingly, incoming interns 
in 2010 should have passed CPX as an essential step to acquire 
a medical license. 7,8 ' CPX is a test that requires great change in 
the medical education system. 9 ' Therefore, it seems necessary 
to evaluate whether the introduction of CPX to the Korean 
medical education system has contributed to the improvement of 
clinicians' quality of performance as expected. 

There have been studies evaluating the impact of CPX on the 
improvement of interviewing skills of medical students. Han et 
al. 10 ' reported a significant correlation between CPX score and 
patient-doctor relationship (r = 0.707). Jang et al. also showed 
a positive correlation between CPX score and the attitude of 
interviewing with patients (r = 0.191). However, no study 
has evaluated the impact of CPX on the clinician's diagnostic 
accuracy of diseases. 

In this regard, we aimed to assess the impact of CPX on 
incoming interns' clinical competency in differential diagnosis 
of headache. Headache has been included in the list of essential 
items of CPX in license examinations for qualified Korean 
medical doctors from the beginning. We hypothesized that 
interns who have been exposed to CPX (CPX-exposed interns) 
would show higher competency compared with interns who have 
not been exposed (CPX non-exposed interns). 

METHODS 



1. Study Subjects 

In order to evaluate the impact of CPX, we compared the 
clinical performance of identifying secondary headache using 
the data of patients with headache who visited emergency 
department (ED) of a tertiary hospital located in Seoul, Korea 
between March 1st, 2009 and January 31st, 2010 ( seen by 69 
CPX non-exposed interns), and between March 1st, 2010 
and January 31st, 2011 (seen by 60 CPX-exposed interns). All 
medical records were reviewed by one of the authors. 

Initially) we reviewed the electronic medical records of 542 
headache patients (290 patients seen by CPX non-exposed interns 
and 252 patients seen by CPX-exposed interns), respectively. 



Among the 542 patients, a total of 211 were excluded from the 
study for the following reasons: headache associated with trauma 
(18 patients), having discharged from the ED on his/her own 
(l patient), medical records lacking information on intern's 
differential diagnosis (57 patients), unavailable information about 
final diagnosis due to not showing at recommended follow-up 
visit (135 patients). Thus, 331 headache cases (181 enrolled m 
2009 and 150 enrolled in 2010) seen by 1 10 interns (60 in 2009 
and 50 in 2010) were finally included in our study. This study was 
approved by the institutional review board of Samsung Medical 
Center. 

2. Study Variables 

We collected data on the demographic and clinical characteristics 
of patients through the review of electronic medical records: age, 
sex, time of ED visit, characteristics of headache, presence of red flag 
signs, past medical history, name of duty intern, list of diagnostic 
impressions for headache given by intern, and final diagnosis for 
headache. In addition, we collected the working information of duty 
doctor for each patient: occurrence of patient hand-off, duration of 
clinical experience of duty intern, and on-duty time of duty intern 
(more than 16 hours or not). 

We collected demographic characteristics of interns by 
reviewing electronic medical records and the nominal lists 
of doctors enrolled in the residency program, including sex, 
exposure to CPX, and enrollment into residency program after 
completing internship training. 

We identified the presence of red flag signs of headache 
according to the Scottish Intercollegiate Guidelines Network 
guidelines on diagnosis and management of headache in adults 
as follows: 12 ' onset of new headache after age >50, change in 
headache frequency or characteristics, headache worsened by 
postural change or Valsalva's maneuver, neck stiffness, focal 
neurologic deficit, associated systemic symptoms, previous 
history of secondary headache or cancer, having risk factors for 
cerebral venous sinus thrombosis, jaw claudication, and being 
infected by human immunodeficiency virus. Focal neurologic 
deficit was defined as the presence of the following features (> 1 ); 
mental status change, disorientation, abnormal findings in cranial 
nerve examination, abnormal findings in motor power or sensory 
examination, increase (> grade 2) or decrease (< grade 2) in deep 
tendon reflex, presence of Babinski or Chaddok sign, abnormal 
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cerebellar function test, and abnormal findings in Romberg's test, 
gait, or tandem gait. 

We categorized the final diagnosis of headache into two 
groups (primary or secondary) according to International 
Headache Society (IHS) classification. 13 ' Secondary headache 
was defined as a headache related with infection, neoplasm, 
vascular disease, cranial nerve pathology, psychiatric cause, 
musculoskeletal cause, iatrogenic, and other ill-defined causes. 
Remaining types of headache were categorized as a primary 
headache. 

We also categorized diagnostic impression by interns into two 
groups (primary headache or secondary headache) according 
to IHS classification, not only on the basis of primary diagnostic 
impression but also on the basis of all listed diagnostic impressions. 
On the basis of primary diagnostic impression, we classified the 
diagnostic impression as secondary headache when the diagnostic 
impression in advance of others was ascertained to be on the lists of 
secondary headaches of IHS classification. On the basis of all listed 
diagnostic impressions, we classified the diagnostic impression as 
secondary headache when any of the listed diagnostic impressions 
was ascertained to be on the lists of secondary headaches of IHS 
classification. 

3. Statistical Analysis 

We compared characteristics between the patients seen 
by CPX non-exposed interns and the patients seen by CPX- 
exposed interns by chi-square test and t-test. We also compared 
the characteristics between the CPX non-exposed interns and the 
CPX-exposed interns. 

In order to assess clinical competency, we calculated sensitivity, 
specificity, positive predictive value, and negative predictive value 
for identifying secondary headache. We repeatedly assessed the 
statistics based on both the primary diagnostic impression and all 
listed diagnostic impressions. 

We defined sensitivity as the proportion of suspected 
secondary headache cases over finally-diagnosed secondary 
headache cases, and specificity as a proportion of suspected 
primary headache cases over finally-diagnosed primary headache 
cases. 14 ' Positive predictive value was defined as a proportion 
of finally-diagnosed secondary headache cases over suspected 
secondary headache cases. 14 ' Negative predictive value was 
defined as a proportion of finally-diagnosed primary headache 



cases over suspected primary headache cases. 

In order to evaluate the association between CPX exposure 
and clinical competency in secondary headache diagnosis, 
we conducted multiple logistic regression analysis with an 
adjustment for provable covariates: sex of intern, 15 ' patient 
handoff, 16 ' enrollment into residency program after completing 
internship training, duration of clinical experience of duty intern, 
duration of working at ED, sex of patient, age of patient, patient 
encounter by intern who was on duty for more than 16 hours, 17 ' 
presence of red flag sign (> l), first-onset headache at the age of 
50 or older, abrupt-onset headache, presence of focal neurologic 
deficits, headache aggravated by postural change or by Valsalva' 
s maneuver, presence of neck stiffness, presence of systemic 
symptoms (fever, dizziness, and general weakness), and history 
of previous secondary headache. Covariates related with clinical 
characteristics of patient were selected because they have been 
commonly considered to be significant clinical factors suggesting 
secondary headache. 12 ' 

We conducted all the analyses under the assumption that all 
measurements were independent of each other. The P-value of 
significance level was set at 0.05 for all statistical assessment. All 
the analyses were performed using SAS ver. 9.3 (SAS Institute 
Inc.,Cary,NC,USA). 

RESULTS 



Table 1 shows general characteristics of interns. The mean 
number ± SD of headache patients per one intern was 3.02 ± 1.97 
for CPX non-exposed interns and 3.00 ± 2.14 for CPX-exposed 
interns. The proportion of male interns and the interns who 
were enrolled in a residency program after completing internship 
training did not differ between the two groups. 

In Table 2, demographic and clinical characteristics of 
patients and related working information of the duty doctor were 
presented. We compared those characteristics between patients 
seen by CPX non-exposed interns and CPX-exposed interns. 
Males occupied a higher proportion among patients seen by CPX- 
exposed interns (46.0%) than among patients seen by CPX non- 
exposed interns (35.3%) with borderline statistical significance. 
The proportion of patients experiencing hand-off of duty intern 
was higher among cases (10.7%) seen by CPX-exposed interns 
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Table 1. General characteristics of interns 



Variable 


CPX non-exposed interns (n = 60) 


CPX-exposed interns (n = 50) 


P-value* 


Male 


29 (48.3) 


22 (44.0) 


0.65 


Case no. 


3.02 ±1.97 


3.00 ±2.14 


0.97 


Enrollment into residency program after internship 


42 (70.0) 


32 (65.3) 


0.60 



Values are presented as number (%) or mean ± SD. 
CPX: clinical performance examination. 

"Obtained by t-test for means and chi-square test for proportions. 



Table 2. General and clinical characteristics of headache cases 



Cases included in study Cases excluded from study 

Seen by CPX Seen by CPX- Seen by CPX SeenbyCPX- 

non-exposed exposed interns P-value* non-exposed exposed interns P-value* 





interns (n = 181) 


(n = 150) 




interns (n= 109) 


(n = 102) 




Age (y) 


47.0 ± 16.7 


49.6 ± 16.5 


0.16 


40.8 ±16.1 


45.0 ±15.9 


0.06 


Male 


64 (35.3) 


69 (46.0) 


0.05 


31 (28.4) 


31 (30.4) 


0.77 


Clinical experience of duty intern > 7 mo 


81 (44.8) 


87 (58.0) 


0.02 


39 (35.8) 


46 (45.1) 


0.17 


Duty intern's clinical experience at ED > 1 mo 


24 (13.3) 


19 (12.7) 


0.87 


6 (5.5) 


12 (11.8) 


0.10 


On-duty time of duty intern (h) 


10.1 ±6.1 


10.5 ±5.9 


0.57 


10.8 ± 7.5 


11.3 ±6.5 


0.54 


On-duty time of duty intern > 16 h 


28 (15.5) 


23 (15.3) 


0.97 


7 (6.4) 


6 (5.9) 


0.87 


Patient hand- off between interns 


9 (5.0) 


16 (10.7) 


0.05 


2 (1.8) 


0 


0.50 


Presence of red flag signs regarding headache 


135 (74.6) 


118(78.7) 


0.38 


41 (37.6) 


49 (48.0) 


0.13 


First-onset headache after 50 years 


27 (14.9) 


22 (14.7) 


0.95 


13(11.9) 


12 (10.8) 


0.84 


Abrupt-onset headache 


67 (39.2) 


56 (40.2) 


0.97 


32 (34.4) 


36 (37.9) 


0.53 


Focal neurologic deficits 


51 (28.2) 


34 (22.7) 


0.25 


15 (13.8) 


12 (12.7) 


0.83 


Aggravated by postural change or Valsalva's 
maneuver 


16 (8.8) 


13 (8.7) 


0.96 


7 (6.4) 


6 (5.9) 


0.87 


Neck stiffness 


8 (4.4) 


3 (2.0) 


0.17 


2 (1.8) 


0 


0.50 


Systemic symptoms 


62 (34.3) 


60 (40.0) 


0.28 


30 (27.5) 


24 (23.5) 


0.51 


Prior secondary headache 


48 (26.5) 


45 (30.0) 


0.48 


8 (7.3) 


11 (10.8) 


0.38 


Cancer history 


10 (5.5) 


7 (4.7) 


0.73 


1 (0.9) 


2 (2.0) 


0.61 



Values are presented as mean ± SD or number (%) . 

CPX clinical performance examination, ED: emergency department. 

*Obtained by t-test, chi-square test, or Fisher's exact test. 



compared to that among cases (5%) seen by CPX non-exposed 
interns (P = 0.05). The proportion of patients seen by duty 
interns with longer clinical experience in internship training (>7 
months) was higher among cases seen by CPX-exposed interns 
(58.0%) compared to that in patients seen by CPX non-exposed 
interns (44.8%) (P = 0.02). For other characteristics, there was 



no difference between the patients seen by CPX non-exposed 
interns and patients seen by CPX-exposed interns. 

When we repeated the comparison in patients who were 
finally excluded from the present study, no significant difference 
was found between the patients seen by CPX-exposed interns 
and patients seen by CPX non-exposed interns for all selected 
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Table 3. Ascertained final diagnosis for headache cases 

Cases seen by CPX Cases seen by 
Variable non-exposed CPX-exposed 

interns (n = 181) interns (n = 150) 



Values are presented as number (%). 

CPX: clinical performance examination, CNS: central nervous system. 
*P-value assessed by chi-square test for the distribution of the final 
diagnosis between two groups of interns > 0.05. 



characteristics. 

Table 3 shows a list of ascertained final diagnoses for 
headaches among patients. In both groups, more than half of 
cases were ascertained as primary headache, and migraine was 
the most prevalent diagnosis. The proportion of patients with 
each ascertained diagnosis did not show a significant difference 
between both groups (P > 0.05 by chi-square test). 

Table 4 shows the estimated accuracy of differential diagnosis 
for headache on the basis of primary diagnostic impression. All 
the statistics assessing diagnostic accuracy tended to be slightly 
higher for the headache cases seen by CPX non-exposed interns 
than that for the headache cases seen by CPX-exposed interns. 

When we repeated the same analysis for estimating the 
diagnostic accuracy for headaches on the basis of all listed 
diagnostic impressions, we observed similar findings (Table 5). 

Table 6 shows the multivariable-adjusted associations 
between CPX exposure and accurate identification of secondary 
headache with an adjustment for various characteristics of 
patients and interns. Exposure to CPX was not associated with 
an improvement in correct identification of secondary headache 
regardless of whether the estimation was conducted on the 
basis of primary diagnostic impression or all listed diagnostic 
impressions. 



Primary headache* 



Migraine 


48 (26.5) 


40 (26.7) 


Tension-type headache 


31 (17.1) 


27 (18.0) 


Cluster headache 


1 (0.6) 


0 


Other type 


7 (3.9) 


4 (2.7) 


Unspecified 


13 (7.2) 


21 (14.0) 


All primary headache 


100 (55.2) 


92 (61.3) 


Secondary headache* 


CNS infection 


13 (7.2) 


4 (2.2) 


CNS neoplasms 


10 (5.5) 


9 (6.0) 


Vascular origin 


21 (11.6) 


15 (10.0) 


Neuralgia 


17 (9.4) 


12 (8.0) 


Musculoskeletal origin 


1 (0.6) 


2 (1.3) 


Psychiatric illness 


4 (2.2) 


2 (1.3) 


Iatrogenic 


7 (3.9) 


4 (2.7) 


Other 


8 (4.4) 


11 (7.3) 


All secondary headache 


81 (44.8) 


58 (38.6) 



Table k. Estimated accuracy* of differential diagnosis for headache on the basis of primary diagnostic impression 



First impression 


Cases seen by CPX non-exposed interns (n = 181) 


Cases seen by CPX-exposed interns (n 


= 150) 




Final diagnosis 






Final diagnosis 




Primary headache 


Secondary headache 


Total 


Primary headache 


Secondary headache 


Total 


Primary headache 


80 (44.2) 


26 (14.4) 


106 (58.6) 


66 (44.0) 


28 (18.7) 


94 (62.7) 


Secondary headache 


20 (11.0) 


55 (30.4) 


75 (41.4) 


26 (17.3) 


30 (20.0) 


56 (37.3) 


Total 


100 (55.2) 


81 (44.8) 


181 (100.0) 


92 (61.3) 


58 (38.7) 


150 (100.0) 


Sensitivity 




67.9 






51.7 




Specificity 




80.0 






71.7 




Positive predictive value 




73.3 






53.6 




Negative predictive value 




75.5 






70.2 





Values are presented as number (%) or %. 
CPX clinical performance examinations. 

'Estimated by sensitivity, specificity, positive predictive value, and negative predictive value. 
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Table 5. Estimated accuracy* of differential diagnosis for headache on the basis of all listed diagnostic impressions 



First impression 


Cases seen by CPX non-exposed interns (n = 181) 


Cases seen by CPX-exposed interns (n 


= 150) 




Final diagnosis 






Final diagnosis 




Primary headache 


Secondary headache 


Total 


Primary headache 


Secondary headache 


Total 


Primary headache 


55 (30.4) 


18 (9.9) 


73 (40.3) 


52 (34.7) 


19 (12.6) 


71 (47.3) 


Secondary headache 


45 (24.9) 


63 (34.8) 


108 (59.7) 


40 (26.7) 


39 (26.0) 


79 (52.7) 


Total 


100 (55.2) 


81 (44.8) 


181 (100.0) 


92 (61.4) 


58 (38.6) 


150 (100.0) 


Sensitivity 




77.8 






67.2 




Specificity 




55.0 






56.2 




Positive predictive value 




58.3 






49.4 




Negative predictive value 




75.3 






73.2 





Values are presented as number (%) or %. 
CPX: clinical performance examinations. 

*Estimated by sensitivity, specificity, positive predictive value, and negative predictive value. 



Table 6. Factors associated with diagnostic values of duty doctors 



Variable 


Primary diagnostic impression 


All listed diagnostic impressions 




Odds ratio (95% CI) 


P-value 


Odds ratio (95% CI) 


P-value 


Exposure to CPX 


0.89 (0.55-1.46) 


0.65 


0.62 (0.37-1.02) 


0.06 


Sex of intern (male) 


0.71 (0.43-1.17) 


0.17 


0.92 (0.54-1.56) 


0.75 


Enrollment of duty intern into residency program next year 


0.72 (0.42-1.24) 


0.24 


0.69 (0.39-1.22) 


0.20 


Duty intern's duration of clinical experience (> 7 mo) 


1.29 (0.77-2.15) 


0.35 


1.08 (0.63-1.85) 


0.79 


Duty intern's clinical experience at emergency department (mo) 


0.39 (0.19-0.81) 


0.01 


0.74 (0.35-1.59) 


0.45 


Sex of patient (male) 


1.78 (1.07-2.94) 


0.03 


1.64 (0.97-2.78) 


0.07 


Age of patient (y) 


0.99 (0.98-1.01) 


0.42 


1.00 (0.98-1.01) 


0.71 


Hand-off between interns 


0.80 (0.32-1.96) 


0.62 


0.50 (0.21-1.20) 


0.12 


Patients encounter by intern who has been on duty > 16 h 


1.02 (0.53-1.98) 


0.95 


1.13 (0.57-2.24) 


0.73 


Presence of red flag signs 


First-onset headache at age >50 y 


0.77 (0.36-1.65) 


0.50 


0.68 (0.32-1.48) 


0.33 


Abrupt- onset headache 


0.80 (0.28-2.30) 


0.24 


0.91 (0.31-2.67) 


0.86 


Focal neurologic deficits 


1.65 (0.88-3.10) 


0.12 


1.31 (0.69-2.51) 


0.41 


Aggravated by postural changes or Valsalva's maneuver 


1.42 (0.60-3.39) 


0.43 


2.22 (0.78-6.30) 


0.13 


Neck stiffness 


7.22 (0.83-62.88) 


0.07 


4.87 (0.56-42.12) 


0.62 


Systemic symptoms 


0.84 (0.51-1.40) 


0.51 


0.73 (0.43-1.23) 


0.24 


Prior secondary headache 


0.93 (0.53-1.61) 


0.78 


0.85 (0.48-1.51) 


0.59 



CI: confidence interval, CPX: clinical performance examination. 

*Estimated by multiple logistic regression model in which all the variables in the above table were put. 
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DISCUSSION 

The present study is the first Korean study to assess the 
impact of CPX on incoming interns' clinical competency by 
comparing the diagnostic accuracy of identifying secondary 
headache between CPX-exposed interns and CPX non-exposed 
interns. Although we hypothesized that CPX-exposed interns 
would show higher competency than CPX non-exposed interns, 
findings of our study did not support our hypothesis. 

Regarding the null effect of CPX on the improvement of 
clinical competency as seen in the present study several possible 
reasons exist. Firstly the interns who were selected for the present 
study had taken CPX for the first time in Korea with very scarce 
experience with CPX. Thus, their clinical competency might 
have been barely influenced by CPX. Secondly the evaluation of 
clinical competency of interns on the basis of headache diagnosis 
might have resulted in underestimation of the impact of CPX. 
A wide range of clinical problems should have been evaluated 
in CPX 8 ' in order to assess clinical competency more accurately 
given that clinical performance consists not only of diagnostic 
accuracy but also procedure skills, patient education, and 
communication skills. 7 ' Thirdly, the impact of CPX might have 
been masked by other unmeasured factors that could have greater 
influence on clinical competency of individual interns. 16 " 21 ' 

Examinations testing clinical competency such as objective 
structured clinical examination (OSCE) or CPX have been 
adopted not only to the Korean medical license examination 
(KMLE) but also to board certifying examinations of plastic 
surgeons, neurologists, and family physicians. There were studies 
that provided evidence for positive aspects of the examinations 
testing clinical competency. In board examinations of plastic 
surgeons, there was a weak association between written test 
scores and OSCE scores (r = 0.32) . 22) In board examinations of 
neurologists, similar findings were revealed. 23 ' A study using a 
questionnaire survey reported that levels of satisfaction regarding 
adopting of CPX into the KMLE were high for physician faculties, 
nurses, patients, and examinees. 9 ' 

Meanwhile, a criticism against the United States Medical 
Licensing Examination step 2 Clinical Skills (USMLE step 2CS), 
one type of CPX, has been raised 24 ' such as inappropriately high 
rate of success (up to 97% ), 25 ' lack of adequate supervision or 
feedback, 24 ' and low cost-effectiveness. 24 ' A study by Clauser et 



al. showed that significant inconsistency may exist regarding 
the assessment of examinees in the USMLE step 2 CS. In 
addition, the USMLE step 2 score reflected residency directors' 
rating for interns less accurately than undergraduate grades 
did. 27 ' Therefore, further studies evaluating the effect of CPX on 
clinical competency of medical students and doctors need to be 
conducted. 

Several factors have been suggested to have an association 
with clinical skill examination scores or clinical performance. 16 " 21 ' 
Ogunyemi and Taylor-Harris 18 ' reported that undergraduate 
written test scores, evaluation grade by faculty, and age of 
examinee were significantly correlated with scores of the 
USMLE step 2, while sex and race were not. In another study in 
Canada, clinical experience, case volume, and socioeconomic 
factors were related with physicians' ability to solve acute health 
problems. 19 ' Reduced workload has also been found to contribute 
to completeness of residents' discharge summaries, 20 ' which 
provided a basis for the Institute of Medicine's recommendation 
that restricts duty hours of a resident to less than 16 hours. 17 ' 
However, we could not find factors that were consistently 
associated with clinical competency in identifying secondary 
headache. 

The present study has several limitations. Firstly, we collected 
data by reviewing the medical record, which may have led to 
biased results. Secondly, one of the authors who knew the 
hypothesis underlying this study performed data collection. 
This un-blinded method of data collection may have resulted 
in overestimation of the beneficial impact of CPX. However, 
this seems unlikely to have occurred, given that rather lower 
diagnostic accuracy was observed for CPX-exposed interns. 
Thirdly, selection bias may have occurred because too many cases 
were excluded from the final analysis and a higher proportion of 
patients with primary headache who need no further evaluation 
or follow up might have been excluded. However, we do not 
believe this distorted our findings, because no significant 
difference was found between the patients seen by CPX- 
exposed interns and the patients seen by CPX non-exposed 
interns for all selected characteristics among the excluded cases. 
Fourthly, it may not be easy to generalize the findings from the 
present study because our study focused on one specific clinical 
entity, headache. Fifthly, although we made an adjustment for 
the enrollment of studied individual interns into residency 
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program after completing internship training, duration of clinical 
experience of duty intern, and duration of working at ED, we 
could not take account for other aspects of individual professional 
competency of interns including KMLE score, performance 
level in medical school, or other unmeasured individual clinical 
abilities due to a lack of information. In addition, as we could not 
control the number of patients seen by each intern equally, the 
competency level of individual interns might also have influence 
the study results. These limitations may have resulted in under- 
or over-estimation of the impact of the Clinical Performance 
Examination on incoming interns' clinical competency in 
differential diagnosis of headache. 

In conclusion, this study on incoming interns found that 
exposure to CPX as a part of the KMLE was not effective for the 
improvement of diagnostic competency for secondary headache. 
However, given the limitations of our study, further evaluations of 
the long-term effect of CPX over more diverse clinical entities are 
needed. 
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